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Objectives: Although vaccination rates have increased, problems still remain in the storage and handling of vaccines. 
This study focused on inspecting actual vaccine storage status and awareness, and comparing them before and after 
education was provided. 

Methods: In the primary inspection, a status survey checklist was completed by visual inspection. A questionnaire on the 
awareness of proper vaccine storage and handling was also administered to vaccine administrators in private medical 
institutions in 4 regions in Gyeongsangbuk-province. One-on-one education was then carried out, and our self-produced 
manual on safe vaccine storage and management methods was provided. In the secondary inspection, the investigators 
visited the same medical institutions and used the same questionnaire and checklist used during the primary inspection. 
The results before and after education were compared, by treating each appropriate answer as 1 point. 
Results: The average checklists score was 9.74 (out of 15 points), which increased significantly after education was 
provided (by 0.84, p<0.001). The participants demonstrated improved practices in recording storage temperatures 
(p=0.016), storing vaccines in the center of the refrigerator (p=0.004), storing vaccines with other medication and non- 
medical items (p=0.031) after education. The average score calculated from the questionnaires was 10.48 (out of 14 
points), which increased after education (by 1.03, p< 0.001). 

Conclusions: This study suggests that vaccine storage practices and awareness are inadequate, but can be partially 
improved by providing relevant education. Repetitive education and policy-making are required to store vaccines safely 
because one-off education and unenforced guidelines offer limited efficacy. 
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INTRODUCTION 

Vaccinations have contributed to decreasing serious 
outbreaks by preventing incidence and propagation of 
contagious diseases in advance. The vaccination 
program is a cost-effective enterprise that provides 
vaccines for preventing the outbreak and propagation of 
vaccine-preventable diseases. Implementing a high 
vaccination coverage rate and the effect of the vaccine 
on the disease will determine whether or not a 
vaccination program can successfully prevent an 
outbreak or the propagation of the disease [1]. Recently, 
many developed and developing countries have pushed 
forward with extending the coverage of required 
vaccinations for every resident their jurisdictions, and 
many countries are making efforts to eliminate the 
geographic and economic limits to vaccination among 
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low and vulnerable classes. In Korea, phased extension 
policy was chosen by government as the coverage plan 
for the required national vaccinations. Thus, the private 
medical institution participation method of the National 
Expanded Program on Immunization had been imple- 
mented since 2005 [2]. Daegu Metropolitan City and 
Gunpo City participated in the pilot project that enforced 
vaccinations for 13 infectious diseases. The objects were 
children 0 to 12 years old and the project was performed 
at private medical institutions; the objects were 
reimbursed for the entire cost of their individual 
inoculations [2]. This developed into the enforcement of 
the National Immunization Program (NIP), since 2009, 
in which a child can receive the part of their inoculation 
cost at most private medical institutions in Korea [3] . If a 
child 12 years old and younger is vaccinated at a 
designated private medical institution, the cost of the 
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Table 1. General characteristics of questionnaire 
subjects and the private medical institutions where 
they work (n=39) 



Contents n (%) 



Questionnaire subjects 
Age (y) 

20-29 1 (2.6) 

30-39 16(41.0) 

40-49 16(41.0) 

50-59 5(12.8) 

>60 1 (2.6) 

Sex 

Male 18(46.2) 

Female 21 (53.8) 
Duty 

Physicians 20 (51 .3) 

Nurses 12(30.8) 

Nurse's aides 3 (7.7) 

Pharmacists 2(5.1) 

Office staff 2 (7.7) 
Years of experience 

<5 2(5.1) 

5-10 12(30.8) 

>11 25(64.1) 
Private medical institutions 
Type of institution 

Hospitals 8 (20.5) 

Pediatric clinics 12(30.8) 

Other miscellaneous clinics 1 9 (48.7) 
No. of vaccination cases (/wk) 

<10 14(35.9) 

10-19 11(28.2) 

20-29 4(10.3) 

30 - 39 4(10.3) 

40 - 49 2(5.1) 

>50 4(10.3) 
No. of vaccines 

1-3 4(10.3) 

4-6 7(17.9) 

7-9 2(5.1) 

10-12 11(28.2) 

>13 15(38.5) 



vaccination is covered by the government on the 
condition that the patient receives vaccinations for 8 of 
the required vaccinations including 22 administrations. 
Implementing such programs for increasing vaccination 
rates has been one of the elements that enabled us to 
prevent contagious diseases; the Korean government has 
been making such policy-related efforts for many years. 

However, in order to ensure success for the vaccination 
program, the efforts not only toward increasing 
vaccination rates, but also toward maintaining the quality 
of vaccines are needed. Due to the fact that most 
vaccines maintain their potency for a short amount of 
time at room temperature, the cold chain is the core of 
the system used for managing the quality and safety of 
vaccines. In addition, the principles related to 
management of vaccines such as installation of 



thermometers, maintenance of appropriate temperatures, 
recording temperatures, location of the vaccines inside a 
refrigerator, defrosting a frozen vaccine, as well as using 
the refrigerator exclusively for vaccine storage must be 
followed in order to finally administer the vaccine at full 
titer. For example, in one localized outbreak of measles 
in the U.S. during the 1970s, illness was associated with 
inoculation at a medical institution where the vaccines 
were stored on a refrigerator door shelf, not in the 
refrigerator s center [1]. In Australia, it was claimed that 
incorrect storage of vaccines could have contributed to 
the resurgence of pertussis [1]. Likewise, the vaccination 
program can be successful in preventing the occurrence 
and spread of illness only if the effect of a vaccine itself 
can be maintained through biological preparation; the 
storage and management of the vaccine are important, 
but the regulations are incomplete, and the distribution 
enterprise and healthcare institution management in the 
field is even more vulnerable [4]. Moreover, there is 
almost no research that has intensively discussed the 
actual state of vaccine storage and management, and 
standardized guidelines along with systematic education 
programs to address these issues are rare. This study 
focused on inspecting actual vaccine storage status and 
awareness at private medical institutions, and compared 
them before and after education was provided. 

METHODS 

I. Study Subjects 

Among the private medical institutions located in 4 
regions (Nam-county in Pohang-city, Yeongju-city, 
Sangju-city, and Mungyeong-city) in the Gyeonsang 
buk-province, we randomly selected 10 medical 
institutions (2 hospitals and 8 clinics that had been 
entrusted with the NIP and that had carried out 
infant/children vaccination in August 2010) from each 
region. Out of a total of 40 private medical institutions, 
we investigated 39 that had agreed in advance over the 
phone to our visit. We requested a visit to those medical 
institutions, but we did not speak about the details such 
as whether or not the medical institutions possess a 
refrigerator exclusively for vaccine storage, or asking for 
records of the refrigerator temperature. 

The investigation was performed over two time 
periods (primary and secondary inspection). The person 
in charge of the vaccinations and a private medical 
institution supervisor of each public health center in the 
jurisdiction of the private medical institutions were 
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visited. Using the checklist, we observed the actual 
status of vaccine storage and handling, and investigated 
the awareness of the person in charge of vaccine 
management using the questionnaire. The questionnaire 
was performed under the principle that stated that the 
person in charge of vaccine management in each private 
medical institution would be the subject of the research, 
but in case there was not a designated person in charge, 
the questionnaire was performed based on the person 
who conducted the most work, on average, relating to 
vaccine management. 

The checklist was prepared for grasping the actual 
state of vaccine storage and handling, and was self- 
produced by referring to existing studies and foreign 
guidelines [1,5,6]. The checklist content includes the 
following: whether or not there was a designated person 
in charge of vaccine management, whether or not the 
person had undergone vaccine management training, 
whether or not the temperature for vaccine storage was 
monitored and recorded, whether or not the appropriate 
temperature (2°C to 8 °C) was maintained, whether or not 
the expiration date was heeded, and whether or not 
equipment of refrigerator reserved for vaccine storage 
and emergency provision. A person in charge of 
vaccination and a private medical institution supervisor 
of a public health center inspected the checklist items. 

The questionnaire for determining the awareness of 
proper vaccine storage and management procedures was 
self-produced by compiling from previous studies and 
foreign guidelines [1,5,6]. After inspecting the checklist, 
the questionnaire was administered by person in charge 
of the public health center vaccinations. Using Likert 
scale for the questionnaire answers, out of five answer 
choices ('strongly agree', 'agree', 'neither agree nor 
disagree', 'disagree', 'strongly disagree'), we let the 
subjects provide answers to the questions according to 
their knowledge regardless of the present storage 
practices. 

After the questionnaire was completed, the people in 
charge of vaccinations at each public health center who 
were also well-informed of knowledge related to vaccine 
storage supervised the one-on-one education of the 
questionnaire subjects. The education manuals used 
were leaflets that were self-produced in advance. 

The content of the education manuals were as follows: 
l)When receiving delivery of vaccine, to maintain the 
cold chain, check whether or not the packaging has been 
destroyed, and check the expiration date. 2) When 
storing a vaccine, the appropriate temperature is 2 °C to 8 
°C. 3) Record the temperature of the refrigerator every 
hour. 4) Preserve a daily record of the temperature over 



the previous two years. 5) Install a refrigerator 
thermometer in the center of the refrigerator. 6) Do not 
store vaccines in areas where a change in temperature 
occurs easily, such as a refrigerator door shelf or the 
front of a refrigerator. 7) Do not keep other kinds of 
drugs or food items in the vaccine refrigerator. 8) Store 
vaccines according to their type for the purpose of easy 
distinction. 9) Expired vaccines need to be discarded 
immediately. 10) The emergency power supply, in case 
of blackout, needs to be checked once a week. 11) The 
opening time of multi-dose vials needs to be recorded. 
12) In case a vaccine is opened, do not store it again and 
do not re-use it the next day. 

The secondary inspection was performed using an 
identical questionnaire and identical checklist by 
revisiting the same medical institution 4 weeks after the 
day of the primary inspection. The subjects of the 
secondary inspection were the same persons as those in 
the primary inspection. 

II. Statistical Analysis 

In order to compare the actual status of vaccine 
storage and handling, the appropriate procedure related 
to each item was treated as 1 point. In the questionnaire, 
where the positive answers were the correct answers, 
'strongly agree', and 'agree' were treated as 1 point, and 
'neither agree nor disagree', 'disagree' and 'strongly 
disagree' as 0 points. Where the negative answers were 
correct, 'disagree' and 'strongly disagree' were treated 
as 1 point, and 'neither agree nor disagree', 'agree', and 
'strongly agree' as 0 points. In order to determine if each 
item of the questionnaire and the checklist had any 
distinction according to the type of medical institution, 
we used the chi-squared test. We added all the scores for 
each item, and we compared the average score of the 
actual status and awareness by type of medical 
institution through one-way ANOVA. 

We compared whether or not there was a difference in 
the actual procedures followed before and after 
education and inspection, according to the items, using 
the Mcnemar's chi-squared test. We compared the total 
score of the primary and secondary inspections using the 
paired r-test. We did a comparison of the awareness 
questionnaire results, according to the items, using the 
Wilcoxon' s signed-rank test. And the total scores of the 
primary and secondary inspection were compared using 
the paired t-test. All statistical analyses were performed 
using SPSS version 17.1 (SPSS Inc., Chicago, IL, USA). 



J Prev Med Public Health 201 2;45(2) 78-89 



Vaccine Storage Practices and Education 81 



Table 2. Vaccine storage and handling status according to itemized checklists by institution types and change 
after education (n=39) 



Vaccine storage and handling practices 


Before 
education 


After education 


p- value 1 


1 . Has a primary vaccine administrator been assigned? 


27 (69.2) 


27 (69.2) 


>0.99 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 
10(83.3) 
11 (57.9) 
0.36 


5 (62.5) 
11 (91.7) 
11 (57.9) 
0.16 




2. Has vaccine storage and handling training been provided to personnel who handle or 


29 (74.4) 


33 (84.6) 


0.29 


administer vaccines? 

Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 
9 (75.0) 
14 (73.7) 
>0.99 


6 (75.0) 
10(83.3) 
17 (89.5) 
0.63 




3. Has the refrigerator its own thermometer? 


34 (87.2) 


36 (92.3) 


0.63 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


7 (87.5) 
12(100) 
15 (78.9) 
0.21 


7 (87.5) 
11 (91.7) 
18 (94.7) 
0.78 




4. Are storage temperatures being recorded? 


27 (69.2) 


34 (87.2) 


0.02 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


5 (62.5) 
11 (91.7) 
11 (57.9) 
0.16 


8 (66.7) 
11 (91.7) 
15 (78.9) 
0.47 




5. Have the recordings of storage temperature been stored for 2 years? 


22 (56.4) 


24(61.5) 


0.50 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


5 (62.5) 

8 (66.7) 

9 (47.4) 
0.63 


6 (75.0) 
8 (66.7) 
10(52.6) 
0.52 




6. Is the refrigerator maintained at the appropriate storage temperature between 2 °c to 8 °c? 


36 (92.3) 


37 (94.9) 


>0.99 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


7 (87.5) 
12(100) 
17(89.5) 
0.58 


7 (87.5) 
12(100) 
18(94.7) 
0.46 




7. Are there any varicella-containing vaccines in the refrigerator? 


2(5.1) 


2(5.1) 


>0.99 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


0 (0.0) 
2(16.7) 
0 (0.0) 
0.13 


0 (0.0) 
2(16.7) 
0 (0.0) 
0.13 




8. Are vaccines being stored in the center of the refrigerator? 


17(43.6) 


26 (66.7) 


0.004 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


2 (25.0) 
5(41.7) 
10 (52.6) 
0.41 


4 (50.0) 
9 (75.0) 
13(68.4) 
0.55 




9. Are other medicines and nonmedical items (e.g., foods) being stored in the refrigerator? 


14(35.9) 


8 (20.5) 


0.03 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


4 (50.0) 
3 (25.0) 
7 (36.8) 
0.56 


3 (37.5) 
1 (8.3) 

4 (21.1) 
0.39 




1 0. Are vaccines being separated by type and expiration date? 


22 (56.4) 


25(64.1) 


0.25 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


4 (50.0) 
8 (66.7) 
10 (52.0) 
0.70 


5 (62.5) 
10(83.3) 
10 (52.0) 
0.24 


_3 


1 1 . Have any of the vaccines passed their expiry dates? 


0 (0.0) 


0 (0.0) 




Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


0 (0.0) 
0 (0.0) 
0 (0.0) 

_3 


0 (0.0) 
0 (0.0) 
0 (0.0) 

_3 


>0.99 


1 2. Are multi-dose vials of vaccine being marked with the date and the time they are first opened? 


29 (74.4) 


27 (69.2) 




Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 
10(83.3) 
13(68.4) 
0.80 


5 (62.5) 
10(83.3) 
12(63.2) 
0.31 
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Table 2. Continued 



Vaccine storage and handling practices 


Before 
education 


After education 


p- value 1 


1 3. Is the refrigerator being used exclusively for vaccine storage? 


3 (7.7) 


3 (7.7) 


>0.99 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

y Value 


3 (37.5) 
0 (0.0) 
0 (0.0) 
u.uuo 


3 (37.5) 
0 (0.0) 
0 (0.0) 

U.UUD 




1 4. Is an emergency power supply available in case of power failure? 


6(15.4) 


5(12.8) 


>0.99 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


3 (37.5) 
0 (0.0) 
3(15.8) 
0.05 


2 (25.0) 
0 (0.0) 
3(15.8) 
0.15 




1 5. Is there also an extra refrigerator in case of emergency? 


27 (69.2) 


29 (74.4) 


0.50 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


3 (37.5) 
11 (91.7) 
13(68.4) 
0.03 


4 (50.0) 
11 (91.7) 
14(73.7) 
0.13 





Values are presented as no. of 'yes' answers (%). 

1 Changes from primary inspection (before education) in 'yes' answers of all 39 institutions were calculated using the McNemar t- test. 

2 Differences according to institution type were evaluated within each inspection by Fisher's exact test of the f- test. 

3 p- value could not be calculated because there were no 'yes' answers. 



RESULTS 

I. General Characteristics 

We report the general characteristics of the 
questionnaire subjects and the relevant private medical 
institution in Table 1. According to the analysis of the 
research subjects, 18 (46.2%) were male, 21 (53.8%) 
were female, and the majority were in their 30s and 40s 
(16 people in each 10-year age range, 41 .0%). The order 
according to duty was as follows: 20 physicians 
(51.3%), 12 nurses (30.8%), 3 nurse aides (7.7%), 2 
pharmacists (5.1%), and 2 members of office staff 
(5.1%). For work experience at a medical institution, 37 
people had worked for more than 5 years, and the 
majority had worked for more than 11 years (25 
institutions, 64.1%). Among the 39 private medical 
institutions investigated, there were 8 hospitals (20.5%), 
12 pediatric clinics (30.8%), and 19 other clinics 
(miscellaneous clinics, 48.7%). As for the average 
number of vaccinated people, in one week, the majority 
of medical institutions (25 institutions, 64.1%) were 
vaccinating less than 20 people, and 14 institutions 
vaccinated less than 10 people (35.9%). Fifteen 
institutions had more than 13 kinds of vaccines (38.5%). 

II. Vaccine storage and Management Practice 

Table 2 shows the checklist items for the vaccine 
storage and management practice inspection, as well as 
the first set of inspection results. Out of the 39 private 
medical institutions, only 3 (7.7%) had refrigerators 



exclusively for storing vaccines, and they were all 
hospital-level medical institutions. Most of the medical 
institutions were maintaining the appropriate 
temperature of the refrigerator for storing vaccines. All 
12 pediatric clinics had maintained an acceptable 
temperature but 2 miscellaneous clinics and 1 hospital 
maintained the refrigerator temperature deviated from 
the acceptable range. There was no significant difference 
among the three types of institutions. In 2 medical 
institutions (5.1%), vaccines that needed to be in the 
freezer were stored inside the refrigerator, and both of 
them were pediatric clinics. A total of 10 (25.6%) 
medical institutions stored multi-dose vials without 
recording the opening dates and as many as 6 
miscellaneous clinics (31.6%) were shown but there 
were no significant differences among the three types of 
institutions. Other than the aforementioned items, some 
of items that showed significant differences according to 
the type of medical institution were whether or not the 
medical institutions had a refrigerator exclusively for 
vaccine storage, whether or not the institution had 
emergency power supplies in case of a blackout, and 
whether or not there were extra refrigerators in case of 
an emergency. There were no pediatric clinics that with 
an emergency power supply in case of a blackout, but 
extra refrigerators were equipped the most in pediatric 
clinics (91 .7%). Overall, item number 11 , whether or not 
the medical institution stored expired vaccines, was 
adhered to the most as none of the 39 institutions were 
storing expired vaccine. 

The score for the checklist of the primary inspection 
was 9.74 ± 0.39 (SE) out of 15 possible total points. The 
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Table 3. Awareness of vaccine storage and handling according to question by institution type and change after 
education (n=39) 



Questions about knowledge of vaccine storage and handling 


Before 
education 


After education 


p- value 1 


1 . A primary vaccine administrator should be assigned 


37 (94.9) 


38 (97.4) 


0.11 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


8(100) 
12(100) 
17 (89.5) 
0.69 


8(100) 
12(100) 
18 (94.7) 

>0.99 




2. A refrigerator for vaccine storage only is needed 


26 (66.7) 


27 (69.2) 


0.13 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


7 (87.5) 
7 (58.3) 
12(63.2) 
0.45 


7 (87.5) 
7 (58.3) 
13(68.4) 
0.45 




3. Vaccines should be stored in a refrigerator designated only for vaccine storage 


25 (64.1) 


25(64.1) 


0.18 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 

7 (58.3) 
12(63.2) 

0.83 


6 (75.0) 

7 (58.3) 
12(63.2) 

0.83 




4. Storing vaccines in a general refrigerator used for other purposes is all right if the temperature is 


6 (15.4) 


5(12.8) 


>0.99 


maintained properly 

Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


4 (50.0) 
0 (0.0) 
2 (10.5) 
0.008 


3 (37.5) 
0 (0.0) 
2 (10.5) 
0.06 




5. Vaccine storage temperatures should be recorded 


32 (82.0) 


36 (92.3) 


0.008 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 
10(83.3) 
16 (84.2) 
0.87 


8(100) 
11 (91.7) 
17(89.5) 

>0.99 




6. The refrigerator for keeping vaccines should be used only for vaccine storage 


36 (92.3) 


38 (97.4) 


0.001 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


7 (87.5) 
12(100) 
17(89.5) 
0.58 


8(100) 
12(100) 
18(94.7) 

>0.99 




7. Vaccines should be kept in groups by type and expiration date 


30 (76.9) 


33 (84.6) 


0.02 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 
11 (91.7) 
13 (68.4) 
0.36 


6 (75.0) 
11 (91.7) 
16(84.2) 
0.73 




8. Within a certain period after opening, multi-dose vials of vaccines should be discarded 


38 (97.4) 


39(100.0) 


0.11 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


8(100) 
11 (91.7) 
19(100) 
0.51 


8(100) 
12(100) 
19(100) 




9. Freeze/thaw cycles are allowed with frozen vaccines 


36 (92.3) 


38 (97.4) 


0.09 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


8(100) 
9 (75.0) 
19(100) 
0.03 


8(100) 
12(100) 
18(94.7) 

>0.99 




1 0. Vaccines should not be exposed to the sun 


37 (94.9) 


39(100.0) 


0.03 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


8(100) 
12(100) 
17 (89.5) 
0.69 


8(100) 
12(100) 

Aft/4 r\r\\ 

19 (100) 

_3 




1 1 . The center of the refrigerator should be equipped with a thermometer 


25(64.1) 


36 (92.3) 


0.003 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 

7 (58.3) 
12(63.2) 

0.83 


8(100) 
11 (91.7) 
17(89.5) 

>0.99 




1 2. For ease of access, vaccines can be stored in the door of a refrigerator 


28(71.8) 


31 (79.5) 


0.002 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


7 (87.5) 
7 (58.3) 
14(73.7) 
0.37 


8(100) 
7 (58.3) 
16(84.2) 
0.09 
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Table 3. Continued 



Questions about knowledge of vaccine storage and handling 


Before 
education 


After education 


p- value 1 


1 3. A generator is necessary in case of blackouts 


23 (59.0) 


27 (69.2) 


0.01 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


6 (75.0) 
5(41.7) 
12(63.2) 
0.28 


6 (75.0) 
8 (66.7) 
13(68.4) 
>0.99 




1 4. Personnel who handle or administer vaccines need training in how to store and handle vaccines 


30 (76.9) 


37 (94.9) 


< 0.001 


Hospitals (n=8) 

Pediatric clinics (n=12) 

Other miscellaneous clinics (n=19) 

p- value 2 


7 (87.5) 

8 (66.7) 
15(78.9) 

0.69 


8(100) 
11 (91.7) 
18(94.7) 

>0.99 





Values are presented as no. of correct answers (%). 

1 Changes from primary administration (before education) in the correct answer for all 39 participants taken together were calculated using the exact test of 
Wilcoxon's signed rank test. 

2 Differences according to institution type were evaluated within each inspection by Fisher's exact test of the f- test. 

3 p- value could not be calculated because all the answers were correct. 



comparison results, according to the kind of medical 
institution, were in the order of pediatric clinics 10.58 ± 
0.39, hospitals 9.62 ±0.62, and miscellaneous clinics 
9.26 + 0.71. There were no significant differences in 
checklist scores according to the type of medical 
institution (Figure 1). 

III. Awareness Investigation 

The questionnaire questions and results for the 
awareness investigation before education are shown in 
Table 3. The question that was answered incorrectly the 
most was that using even a general refrigerator is 
appropriate for storing vaccines as long as the 
temperature is maintained properly (item 4), and only 6 
people (15.4%) out of all subjects answered 'strongly 
disagree' or 'disagree'. There was a significant 
difference according to the type of medical institution, 
and in particular, all the subjects of the pediatric clinics 
had the wrong answer for that question. Items answered 
correctly included 'a refrigerator for vaccine storage 
only is needed' (item 2), 'vaccines should be stored in a 
refrigerator designated only for vaccine storage' (item 
3), which had 26 people (66.7%) and 25 people (64.1%), 
respectively. Though there was no significant difference 
according to the types of medical institutions there was a 
low awareness demonstrated in pediatric clinics. There 
were 3 people (7.7%), all from pediatric clinics, who 
knew that freeze and thaw cycles are allowed with 
frozen vaccines (item 9), and the difference according to 
types of medical institutions was significant. The 
question with the highest number of subjects that 
showed proper awareness was that multi-dose vials of 
vaccines should be discarded within a certain period 
after opening (item 8) . 



15 
14 
13 
12 
11 
0) 10 

8 9 

" 8 

03 

7 

CD 

CD 6 

< 5 
4 

3 
2 



■ Before education □ After education 



i ** *** 

MM 

Hospitals Pediatric clinics Other miscellaneous Total 
clinics 

Figure 1. Average score of vaccine storage and 
handling status and change after education by 
institution type. 

Bars represent the mean ± SE, and a perfect score is 15 points; 4 weeks 
after education, pediatric clinics, other miscellaneous clinics, and the three 
types in aggregate had increased average scores by 0.75, 0.95, and 0.84 
points, respectively, compared with before education (*p<0.05, **p<0.01 , 
***p<0.001). There were no significant differences among institution types in 
scores either before or after education. 

The average awareness score of the primary inspection 
was 10.48 ± 0.34 (SE). Based on the results according 
to the type of medical institution, the order was as 
follows: hospitals 11.75 ± 0.86, miscellaneous clinics 
10.36 ±0.49, and pediatric clinics 9.83 ± 0.50. The 
resulting value was highest overall for the hospitals but 
there was no significant difference (Figure 2). 

IV. Comparison of Before and After Education 

We compared the actual status of vaccine storage 
between the primary and secondary inspection. Medical 
institutions that record the refrigerator temperature (item 
4) increased significantly from 27 institutions (69.2%) to 
34 institutions (87.2%). The number of medical 
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■ Before education □ After education 




Hospitals Pediatric clinics Other miscellaneous Total 
clinics 



Figure 2. Average score of awareness of vaccine 
storage and change after education by institution 
type. 

Bars represent the mean + SE, and a perfect score is 1 4 points; 4 weeks 
after education, pediatric clinics, other miscellaneous clinics, and the three 
types in aggregate had increased average scores by 1 .25, 1 .00, and 1 .03 
points, respectively, compared with before education (**p<0.01 , ***p<0.001 ). 
There were no significant differences among institution types in scores either 
before or after education. 

institutions that stored vaccines in the center of the 
refrigerator (item 8) increased significantly from 17 
institutions (43.6%) to 26 institutions (66.7%), and 
institutions that stored vaccines with other kinds of 
medications or food significantly decreased from 14 
institutions (35.9%) to 8 institutions (20.5%) (Table 2). 
The average secondary inspection score of the checklist 
for all of the private medical institutions together was 
10.58 ± 0.34 (SE), and it significantly increased by 0.84 
points compared to the primary inspection (p< 0.001) 
(Figure 1). For each type of medical institution 
considered separately, the score after education for 
hospitals increased by 0.75 (p = 0.320), pediatric clinics 
by 0.75 (p = 0.032), and miscellaneous clinics by 0.95 
(p =0.002) (Figure 1). 

Awareness after education, compared to before the 
education, significantly improved in items 5-7, which 
were about the completion of temperature records, 
exclusive storage of vaccines, and selective storage of 
vaccines according to the type and expiration date, as 
well as items 10-14 which were light-proof storage, 
installation of a thermometer in the center of the 
refrigerator, storing the vaccines in the center of a 
refrigerator, installation of an independent generator, and 
educating people in charge of vaccines. The awareness 
score of all the medical institutions after education was 
11.51 ± 0.28 (SE), and it increased by 1.03 points 
compared to before the education (p< 0.001) (Figure 2). 
According to the types of medical institutions, hospitals 
increased by 0.75 points (p =0.197), pediatric clinics by 
1.25 points (p =0.006), and miscellaneous clinics by 



1 .00 point (p = 0 .005) (Figure 2) . 

DISCUSSION 

If a group of people has a high immunity level, even 
with some susceptible people in the group, there is a 
small possibility that an outbreak can occurs. In 
particular, if a human is the only host organism, 
increasing the level of immunity of a group can eradicate 
an infectious disease [7]. Using this method, smallpox 
has been the only disease humankind has completely 
eradicated up to the present time, but the number of 
eradicated diseases can be increased through vaccination 
programs. Diseases such as polio and diphtheria are at 
the level of being almost entirely eradicated [2]. 
However, until the present time, the focus has been on 
coverage rates of vaccines, and the problems related to 
the quality of handling after the production of a vaccine 
have received little attention. Then, Lerman and Gold [8] 
in 1971 and Krugman et al. [9] in 1974 reported that one 
plausible factor that resulted in outbreak of measles in 
children with a vaccination history was improper storage 
of vaccines and thus the effect of vaccines could have 
been reduced. Therefore, problems related to storage and 
management of vaccines captured people's attention. 
However, reviewing the following studies, recognition 
related to the importance of vaccine storage has not been 
well established yet, and the actual post-production 
handling conditions have been diverse [5,10-15]. 
Steinmetz et al. [10] investigated the actual conditions 
for storing live measles, mumps, and rubella virus 
vaccines. In 1981, 60% of the investigated medical 
institutions had the refrigerator temperature at higher 
than 8 °C, and they reported that only 40% were storing 
vaccines at the recommended location in the refrigerator. 
In the research performed by Hunter [11], only 3 
medical institutions out of 36 inspected had installed a 
thermometer inside the refrigerator. In Bishai et al. [12] 
22% of refrigerators had an inside temperature that was 
higher than 8 °C, and 46% had stored the vaccines on the 
refrigerator door shelf. Thakker and Woods [13] stated 
that out of 40 subjects who answered their questionnaire, 
only 16 knew the correct vaccine storage method. Yuan 
et al. [5] investigated both knowledge and actual 
conditions related to the storage and handling of 
vaccines from a primary care physician's office, and 
only 6% of participants provided correct answers to all 4 
of the questions, which showed the necessity of 
awareness improvement. At the same time, they stated 
that only 10% of the investigated medical institutions 



JPrev Med Public Health 2012;45(2)78-89 



86 Saerom Lee et al. 



had installed thermometers inside their refrigerators, and 
about one-third of them had temperatures outside the 
recommended range. Just as mentioned earlier, despite 
the fact that many studies have reported on the 
importance and seriousness of vaccine storage and 
management conditions, the efforts of health personnel 
including policy-decision-making institutions to remedy 
the situation have been insufficient. 

In Korea, the distribution system for vaccines in 
private medical institutions is one in which the hospitals 
and clinics have direct deals with either the 
manufacturing company or sales company, or are being 
provided by the wholesale markets; it is very important 
to properly maintain the cold chain throughout all these 
processes. The cold chain, also known as the low- 
temperature distribution system, is a delivery system that 
maintains a low temperature from the time of production 
until the time of supply, and is the core principle of 
management of vaccine quality and safety. Although this 
study was not able to widen the research to include the 
whole distribution process including the manufacturing 
company, sales company, and the wholesale market, the 
authors apprehended the actual conditions of vaccine 
storage and handling in private medical institutions 
located in 4 regions in Gyeongsangbuk-province that 
performed infant and child vaccinations. 

There are two ways in which the titer of the final 
administrated vaccine can be directly decreased due to 
an incorrect storage method: one way is to use vaccines 
that were not stored at the appropriate temperature, and 
the other is to use vaccines whose expiration date has 
passed [6,9]. The latter can include using opened multi- 
dose vials of vaccine that have been exceeded a certain 
acceptable time period for use, after they have been 
opened. The questionnaire items that most directly 
reflect the two abovementioned ways are 6, 7, 11, and 
12. The checklist for actual status of item 6 is the 
measurement of direct storage temperature, and item 7 
showed that vaccines were stored at a wrong 
temperature. Items 11 and 12 reflect whether it is 
possible to use vaccines anytime from slightly to 
extremely past their expiration date. As a result of the 
primary investigation, 3 medical institutions were not 
maintaining the refrigerator temperature at 2 °C to 8 °C, 
and there were even as many as 2 institutions that had 
stored vaccines in the refrigerator that were supposed to 
be stored in the freezer. The former institutions were all 
hospitals or miscellaneous clinics, whereas, the latter 
were all pediatric clinics. However, we cannot determine 
whether a certain type of medical institution had an 
actual serious problem related to vaccine storage based 



only on the results mentioned above, because the 
temperature of the refrigerators were measured only 
once, and vaccines that have to be kept inside the freezer 
like varicella-containing vaccines are administrated 
frequently at pediatric clinics. In addition, this is because 
there was a practical limit on the number of investigation 
subjects. None of the medical institutions that 
participated in the investigation that stored expired 
vaccines, but 25.6% of the institutions did not record the 
opening date of multi-dose vials of vaccine. Out of those 
25.6% institutions, there were relatively more 
miscellaneous clinics, but the difference among types of 
institutions was not significant. The important thing to 
keep in mind is that administrating vaccines without 
abiding by items 6, 7, 11, and 12 can possibly lead to 
failure of formation of the antibody. 

Among the private medical institutions investigated, 3 
of them were furnished with refrigerators exclusive for 
vaccine storage, but most of them were storing vaccine 
in typical refrigerators. For storing vaccines, stand-alone 
freezers and refrigerators without freezers are preferred 
because maintaining the temperature is easier than with a 
combination unit [6]. Even if the equipment has a 
refrigerator and freezer combined, each compartment 
must have an exterior door and temperature-maintaining 
equipment. The type of refrigerator that should not be 
used for storing vaccine is a dormitory-style refrigerator. 
This type of refrigerator has access to the refrigerator 
and freezer at the same time without a separate door, 
which forms one small unit. The cooling coil is normally 
located in the freezer section of this type of refrigerator, 
and the temperature sensor is not sensitive enough to 
fulfill its intended function. In particular, once it 
decreases the temperature, many times the temperature 
of the refrigerator falls below 0°C . Even if the vaccine is 
stored for a short period of time, it is necessary to use a 
refrigerator that is appropriate for storing vaccine to 
prevent the vaccine from being exposed to a temperature 
that is either too high or too low, thus, preventing the 
spoiling of the vaccine [6] . 

After the primary inspection, it was found that only 
7.7% of the medical institutions were storing only 
vaccines in the refrigerator. It is unacceptable to store 
other medical products or food in a place where vaccines 
are stored, because other drugs of foods can pollute 
vaccines. In addition, there would be more frequent 
opening and closing of the door, which could make it 
difficult to maintain the appropriate temperature for the 
vaccines [6]. All the better, maintaining the appropriate 
temperature is very important, and thus regular checking 
of the temperature is very important. However, the 
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principle of keeping the vaccine in the center of a 
refrigerator, the location where the temperature is rather 
consistent, was adhered to by less than half of all the 
institutions inspected (43.6%). Refrigerators without a 
thermometer were found in as many as 12.8% of the 
institutions, and 69.2% of institutions recorded the 
storage temperature regularly. In addition, only 15.4% 
had an available emergency power supply in case of a 
blackout. From these facts, it was found that many 
medical institutions were exposing vaccine to unstable 
temperatures and that caution about this was lacking 
[12]. Not only can this decrease the titer of vaccine, but 
it can also lead to discarding vaccine and re- vaccination 
due to failure of antibody formation, which would lead 
to decrease in the credibility of the vaccine efficacy to 
the patients. 

The average score from the checklist was 64.9%, 
which was a rather low in percentage, so we discovered 
that the overall actual condition needs to be ameliorated. 
Although there was not a significant difference 
according to the type of institution, pediatric clinics had 
a slightly higher score than the other institutions. 
Because pediatric clinics provide many patients with 
vaccinations, there was a high possibility that the storage 
conditions would have been good [1] . 

The average score from the awareness questionnaire 
was 74.9%, and the figure was relatively higher than the 
storage condition average score. However, based on the 
point of view that one needs to have a correct 
understanding for every item in order to handle vaccines 
properly, this awareness score is not satisfactory. Yuan et 
al. [5], also considered answering all the questions 
correctly as very important. Pediatric clinics had the 
lowest awareness score, which was contradictory to the 
actual conditions of investigation, but there was no 
significant difference among types of institutions. 

When we compared the checklist score and 
questionnaire score before and after education, we found 
that the actual conditions of the overall vaccine storage 
and awareness had improved. All of the 3 types of 
institutions improved in both actual conditions and 
awareness; the pediatric clinics and other miscellaneous 
clinics showed significant differences. 

Most of the checklist items were possible to improve 
mostly through awareness improvement and individual 
effort, except for whether or not the institution was 
equipped with refrigerators exclusive for vaccines, and 
whether or not an emergency power supply was present. 
After education was provided, we saw that the actual 
conditions significantly improved for items such as 
completion of recording vaccine storage temperature, 



storing the vaccine in the center of the refrigerator, and 
storing vaccines separately from other drugs or food. 
Although actual conditions related to vaccine storage 
such as a change in behavior of the vaccine storage 
manager could be further improved with a greater effort, 
the items that were not possible to improve could have 
been due to the limitations of one-off education. 
Repeated education for the improvement of behavior 
needs to be enforced. In addition, if a periodic reminder 
such as a written document or an electronic document 
was sent to an institution, along with enforcement of 
surveillance and feedback, problems related to vaccine 
storage or management could be largely solved. These 
items need to be supported with policies, because their 
fulfillment would significantly decrease without the 
support of policies. 

In awareness questionnaire, compared to before 
education, after education several items significantly 
improved, including awareness of filling out the vaccine 
storage temperature records, storing vaccines separately 
from other drugs or food, storing vaccines according to 
type and expiration date, storage in light-proof 
conditions, installation of a thermometer in the center of 
the refrigerator, not storing vaccines on the refrigerator 
door shelf, the necessity for having an independent 
generator available in preparation for a blackout, and 
awareness of the necessity to educate the person in 
charge of vaccine management. Out of these items in the 
questionnaire, awareness about storing vaccines 
according to type and expiration date, and operation of 
an emergency power supply had significantly improved, 
but improvement in actual practice was meager. When 
the subjects were aware of those items, there were a few 
cases where conditions would have allowed those items 
to be implemented and practiced, but the reality was that 
in most of the institutions, resources were insufficient for 
purchasing a refrigerator exclusively for vaccines or 
possessing an independent generator. Refrigerators 
designated specifically for vaccine storage cost 9 million 
won (about $US 9000) per 780 liters, and realistically 
speaking, private medical institutions whose quantity of 
vaccinations is less than 10 cases per week cannot afford 
vaccine refrigerators. Therefore, policy should not stop 
at implementing regular inspections and surveillance, 
but it is imperative that it also include a governmental 
system that provides resources for private medical 
institutions to purchase refrigerators for vaccine storage. 
Most of the problems can be improved by individual 
effort, but for those problems that cannot be solved too 
easily, as mentioned earlier, we may assume that it is due 
to the limits of a one-off education of the person in 
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charge of vaccine management because it may not 
actually modify behavior itself. Therefore, the results of 
this research enabled us to confirm that repetitive 
education, centralized management, and surveillance are 
important. 

Self-produced educational leaflets provided at the 
same time as one-on-one education included only the 
basic knowledge required of people in charge of vaccine 
management. In Korea, the situation is dire enough that 
we need to produce a standardized manual related to 
vaccine storage and management. In the U.S., the 
Centers for Disease Control and Prevention publicly 
announced their newest standardized guidelines for 
vaccine storage and handling on their website for users 
to access easily. On the other hand, in Korea it is difficult 
to find a standardized guidelines written in Korean 
except for a couple of sections from the Handbook for 
safe use of vaccine [16]. 

There are some limitations to this study. First, the 
private medical institutions could have had a bias, 
because we called them on the phone prior to the visit 
asking for their permission, and although they might not 
have known the detailed items related to the inspection, 
they knew we would visit them on the matter of 
vaccination. Second, we only targeted private medical 
institutions that were controlled by 4 public health 
centers in a particular region, which means that the 
results may not be generalizable to the other regions of 
Korea. Third, although public health centers administer a 
large number of vaccinations, we performed the study in 
collaboration with people in charge of the public health 
center; thus, we did not include them as study targets, 
but it is necessary to broaden the range and the subjects 
in future studies. Fourth, measuring temperatures inside 
the refrigerators was not done repeatedly; it was only 
performed once. 

Despite these limitations, this study demonstrated that 
proper awareness and actual practices related to vaccine 
storage and management are rare in Korea. From a 
narrow perspective, the goal of this study was to 
improve the awareness and practices of health care 
providers in some regions of Gyeongsangbuk-province. 
By extension, this study is meaningful in the sense that it 
proposes establishing policies to enforce repeated 
education, developing standardized manual, and 
performing periodic management inspection by 
communicating the results to the people in the public 
health policy decision-making process throughout the 
country. In conclusion, vaccination is the most effective 
and the strongest method ever developed for humans to 
prevent and eliminate contagious diseases, and a system 



needs to be established for proper vaccine storage and 
quality control to increase its efficacy. 
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